Lecture 07 H testing and simple tests II

Bill Perry

Lecture 6 - A Brief review

  • Hypotheses
  • 1- and 2-sided T tests
  • Assumptions of parametric tests

Lecture 7 overview

What we will cover today:

  • What are the assumptions again and how do you assess them
  • What to do when assumptions fail
    • Robust tests
    • Rank-based tests
    • Permutation tests

Lets work with the Lake Trout data as the weights are pretty cool and the assumptions may or may not hold

This is easily translated into any of the other dataframes you might want to use

lake trout

# Install packages if needed (uncomment if necessary)
# install.packages("readr")
# install.packages("tidyverse")
# install.packages("car")
# install.packages("here")

# Load libraries
library(car)          # For diagnostic tests
library(patchwork)
library(tidyverse)    # For data manipulation and visualization
# the stuff above controls the output and is also set at the top so dont need here
# Load the pine needle data
# Use here() function to specify the path
lt_df <- read_csv("data/lake_trout.csv")

# Examine the first few rows
head(df)
                                              
1 function (x, df1, df2, ncp, log = FALSE)    
2 {                                           
3     if (missing(ncp))                       
4         .Call(C_df, x, df1, df2, log)       
5     else .Call(C_dnf, x, df1, df2, ncp, log)
6 }

Parametric versus non-parametric tests

T-tests are parametric tests

  • Parametric tests:
    • specify/assume probability distribution from which parameters came
    • Basic assumptions of parametric t-tests:

      • Random sampling

      • Normality

      • Equal variance

      • No outliers

  • Non-parametric tests: no assumption about probability distribution
    • Mukasa et al 2021 DOI: 10.4236/ojbm.2021.93081

Assumptions of parametric tests

  • If assumptions of parametric test violated, test becomes unreliable
  • This is because test statistic may no longer follow distribution
  • Most parametric tests robust to mild/moderate violations of below assumptions

Assumptions of parametric tests

  • Basic assumptions of parametric t-tests:

    • Random sampling

    • Normality

    • Equal variance

    • No outliers

  • Random sampling:

    • samples are randomly collected from populations; part of experimental design

    • Necessary for sample -> population inference

<>

Assumptions of parametric tests

Basic assumptions of parametric t-tests:

  • Normality
  • equal variance
  • random sampling
  • no outliers
  • Lets do the above for one lake - NE 12 as if we were going to do a one sample T Test
    • we need to make a new dataframae with NE 12 data only called ne12_data
    • how do you do this?
  • Normality: Samples from normally distributed population
    • Graphical tests: histograms, dotplots, boxplots, qq-plots
    • “Formal” tests: Shapiro-Wilk test - sometimes not useful

Assumptions of parametric tests

Basic assumptions of parametric t-tests:

  • Normality
  • equal variance
  • random sampling
  • no outliers
  • Lets do the above for one lake - NE 12 as if we were going to do a one sample T Test
    • we need to make a new dataframae with NE 12 data only called ne12_data
    • how do you do this?
  • Normality: Samples from normally distributed population
    • Graphical tests: histograms, dotplots, boxplots, qq-plots
    • “Formal” tests: Shapiro-Wilk test - sometimes not useful
[1] "Null hypothesis is that data is normally distributed"

    Shapiro-Wilk normality test

data:  ne12_data$length_mm
W = 0.94528, p-value = 1.56e-09

Assumptions of parametric tests

Basic assumptions of parametric t-tests:

  • Normality
  • equal variance
  • random sampling
  • no outliers

Equal variance: samples are from populations with similar degree of variability

  • Graphical tests: boxplots
  • “Formal” tests: F-ratio test
  • When samples sizes equal
    • Parametric tests most robust to violations of normality
    • Less so for equal variation assumptions
length_plot <- ne12_data %>% ggplot(aes(x=lake, y = length_mm)) +geom_boxplot() 
mass_plot <- ne12_data %>% ggplot(aes(x=lake, y = mass_g)) +geom_boxplot()
length_plot + mass_plot + plot_layout(ncol=1)

Assumptions of parametric tests

  • Basic assumptions of parametric t-tests: - Normality
-   equal variance

-   random sampling

-   no outliers
  • No outliers: no “extreme” values that are very different from rest of sample
    • Graphical tests: boxplots, histograms
    • “Formal tests”: Grubb’s test - no one really does this
    • Note: outliers a problem for non-parametric tests as well
ne12_histo_plot + ne12_box_plot + plot_layout(ncol = 1)

Nonparametric test

  • What if T Test assumptions fail?
  • Alternative tests, with more relaxed assumptions, are available
  • In which case would you use the following tests?
    • Welch’s t-test: when distribution normal but variance unequal
    • Mann-Whitney-Wilcoxon test: when distribution not normal and/or outliers are present (but both groups should still have similar distributions and ~equal variance)
    • Permutation test for two samples: when distribution not normal (but both groups should still have similar distributions and ~equal variance)

< –

ne12_histo_plot+ne12_box_plot + ne12_qq_plot + plot_layout(ncol=1)

Assumptions of parametric tests

  • QQ-plots: tool for assessing normality
    • On x- theoretical quantiles from SND
    • On y- ordered sample values
    • Deviation from normal can be detected as deviation from straight line
isl_ne12_df <- lt_df %>% filter(lake %in% c("NE 12", "Island Lake"))

ne12_island_box_plot <- isl_ne12_df %>% 
  ggplot(aes(x=lake, y = mass_g, color=lake)) +
  geom_boxplot()+
  theme_minimal()
# ne12_island_box_plot

ne12_island_qq_plot <- isl_ne12_df %>% 
  # filter(lake =="NE 12") %>%
  ggplot(aes(sample = mass_g , color=lake)) +
  stat_qq() +
  stat_qq_line() +
  labs( x = "Theoretical Quantiles", y = "Sample Quantiles") +
  theme_minimal() 

ne12_island_box_plot +ne12_island_qq_plot+plot_layout(guides="collect")

Assumptions of parametric tests

  • In some cases, data can be mathematically “transformed” to meet assumptions of parametric tests
  • this can be done in r and usually involves
    • log10 transformations
    • square root transformations
    • and many others… I will have a description soon

<>

source

Robust tests

  • Welch’s t-test
    • common “robust” test for means of two populations
    • Robust to violation of equal variance assumption, deals better with unequal sample size
    • Parametric test (assumes normal distribution)
    • Calculates a t statistic but recalculates df based on samples sizes and s
log_ne12_island_box_plot <- isl_ne12_df %>% 
  ggplot(aes(x=lake, y = log10(mass_g), color=lake)) +
  geom_boxplot()+
  theme_minimal()


ne12_island_box_plot +log_ne12_island_box_plot +plot_layout(guides="collect")

Robust tests

  • Lets compare a parametric T-Test to a Welch’s t-test

    • T-Test is:
      • t.test(y1, y2, var.equal = TRUE, paired = FALSE)
    • Welch’s T-Test is:
      • t.test(y1, y2, var.equal = FALSE, paired = FALSE)
[1] "Standard t-test results for lenght_mm:"

    Two Sample t-test

data:  mass_g by lake
t = 14.181, df = 330, p-value < 2.2e-16
alternative hypothesis: true difference in means between group Island Lake and group NE 12 is not equal to 0
95 percent confidence interval:
 2266.304 2996.360
sample estimates:
mean in group Island Lake       mean in group NE 12 
                3165.0000                  533.6677 
[1] "Welch's t-test results for lenght_mm:"

    Welch Two Sample t-test

data:  mass_g by lake
t = 5.1368, df = 9.0578, p-value = 0.0006016
alternative hypothesis: true difference in means between group Island Lake and group NE 12 is not equal to 0
95 percent confidence interval:
 1473.676 3788.989
sample estimates:
mean in group Island Lake       mean in group NE 12 
                3165.0000                  533.6677

Rank based tests

Rank-based tests: no assumptions about distribution (non-parametric)

  • Ranks of data: observations assigned ranks, sums (and signs for paired tests) of ranks for groups compared

  • Mann-Whitney U test common alternative to independent samples t-test

  • Wilcoxon signed-rank test is alternative to paired t-test

  • Assumptions: similar distributions for groups, equal variance
  • Less power than parametric tests
  • Best when normality assumption can not be met by transformation (weird distribution) or large outliers

<

[1] "Mann-Whitney U test results length:"

    Wilcoxon rank sum test with continuity correction

data:  mass_g by lake
W = 3205.5, p-value = 9.506e-08
alternative hypothesis: true location shift is not equal to 0

Permutation tests

  • Permutation tests based on resampling: reshuffling of original data

  • Resampling allows parameter estimation when distribution unknown, including SEs and CIs of statistics (means, medians)

  • Common approach is bootstrap: resample sample with replacement many times, recalculate sample stats

  • Use the perm package

  • Ho: µA = µB

  • Ha: µA ≠µB

  • Calculates the difference ∆ in means between two groups

<!

Permutation tests

  • Randomly reshuffle observations between groups (keeping nNE 12=323 and nIsland=10), calculate ∆
  • Repeat >1,000 times
  • Record proportion of the different means i
  • This is equivalent to p-value and can be used in “traditional” H test framework
  • For a graphical explanation:

Permutation tests

  • In R (using ‘perm’ package):
  • Assumptions: both groups have similar distribution; equal variance
library(perm) 

# Prepare data for permutation test
ne12_perm_data <- isl_ne12_df %>% 
  filter(lake == "NE 12") %>% 
  pull(length_mm)

# Randomly sample exactly 25 observations from NE 12 (set seed for reproducibility)
set.seed(123)
ne12_perm_data <- sample(ne12_perm_data, size = 25, replace = FALSE)

island_perm_data <- isl_ne12_df %>% 
  filter(lake == "Island Lake") %>% 
  pull(length_mm)

# Calculate the observed difference in means
observed_diff <- mean(ne12_perm_data, na.rm = TRUE) - mean(island_perm_data, na.rm = TRUE)

# Perform permutation test for difference in means using perm package
permTS(ne12_perm_data, island_perm_data, 
       alternative = "two.sided", 
       method = "exact.mc", 
       control = permControl(nmc = 10000))

    Exact Permutation Test Estimated by Monte Carlo

data:  GROUP 1 and GROUP 2
p-value = 2e-04
alternative hypothesis: true mean GROUP 1 - mean GROUP 2 is not equal to 0
sample estimates:
mean GROUP 1 - mean GROUP 2 
                    -333.08 

p-value estimated from 10000 Monte Carlo replications
99 percent confidence interval on p-value:
 0.000000000 0.001059383